Derivation of linear parameter in cross-component video coding

ABSTRACT

A method of video processing includes deriving, for a conversion between a chroma block of a video and a bitstream representation of the video, parameters of a cross-component linear model by using downsampled luma samples that are generated from N above neighboring lines of a collocated luma block of the chroma block using a downsampling filter, where N is a positive integer; and performing the conversion using a predicted chroma block generated using the cross-component linear model.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/CN2020/125796, filed on Nov. 2, 2020, which claims the priority toand benefits of International Patent Application No. PCT/CN2019/115034,filed on Nov. 1, 2019. For all purposes under the law, the entiredisclosures of the aforementioned applications are incorporated byreference as part of the disclosure of this application.

TECHNICAL FIELD

This document is related to video and image coding and decodingtechnologies.

BACKGROUND

Digital video accounts for the largest bandwidth use on the internet andother digital communication networks. As the number of connected userdevices capable of receiving and displaying video increases, it isexpected that the bandwidth demand for digital video usage will continueto grow.

SUMMARY

The disclosed techniques may be used by video or image decoder orencoder embodiments for performing encoding or decoding usingcross-component linear model prediction.

In one example aspect, a method of processing video is disclosed. Themethod includes deriving, for a conversion between a chroma block of avideo and a bitstream representation of the video, parameters of across-component linear model by using downsampled luma samples that aregenerated from N above neighboring lines of a collocated luma block ofthe chroma block using a downsampling filter, where N is a positiveinteger; and performing the conversion using a predicted chroma blockgenerated using the cross-component linear model.

In another example aspect, a method of processing video is disclosed.The method includes determining, for a conversion between a video regionof a component of a video and a bitstream representation of the video, amaximum allowed block size for a video block coded using a transformskip mode; and performing the conversion based on the determining.

In another example aspect, a method of processing video is disclosed.The method includes performing a conversion between a video comprisingvideo blocks and a bitstream representation of the video according to afirst rule and a second rule, wherein a transform skip coding tool isused for coding a first portion of the video blocks, wherein a transformcoding tool is used for coding a second portion of the video blocks,wherein the first rule specifies a maximum allowed block size for thefirst portion of the video blocks and the second rule specifies amaximum allowed block size for the second portion of the video blocks,and wherein the maximum allowed block size for the first portion of thevideo blocks is different form the maximum allowed block size for thesecond portion of the video block.

In another example aspect, a method of processing video is disclosed.The method includes performing a conversion between a video comprisingone or more chroma blocks and a bitstream representation of the video,wherein the bitstream representation conforms to a format rule thatspecifies that whether a syntax element to indicate usage of a transformskip tool is included in the bitstream representation depends on amaximum allowed size for a chroma block that is coded using thetransform skip tool.

In another example aspect, a method of processing video is disclosed.The method includes performing a conversion between a video comprisingone or more first video blocks of a first chroma component and one ormore second video blocks of a second chroma component and a bitstreamrepresentation of the video, wherein the bitstream representationconforms to a format rule that specifies to use a syntax element thatjointly indicates availability of a transform skip tool for coding theone or more first chroma blocks and the one or more second chromablocks.

In another example aspect, the above-described method may be implementedby a video encoder apparatus that comprises a processor.

In yet another example aspect, these methods may be embodied in the formof processor-executable instructions and stored on a computer-readableprogram medium.

These, and other, aspects are further described in the present document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows Nominal vertical and horizontal locations of 4:2:2 lumaand chroma samples in a picture.

FIG. 1B shows an example of a video encoder.

FIG. 2 shows examples of 67 intra prediction modes.

FIG. 3 shows examples of horizontal and vertical traverse scans.

FIG. 4 shows examples of locations of the samples used for thederivation of α and β.

FIG. 5 shows example of dividing a block of 4×8 samples into twoindependently decodable areas.

FIG. 6 shows an example order of processing of the rows of pixels tomaximize throughput for 4×N blocks with vertical predictor.

FIG. 7 shows an example of a low-Frequency Non-Separable Transform(LFNST) process.

FIG. 8 shows an example of neighbouring chroma samples and downsampledcollocated neighbouring luma samples used in the derivation of CCLMparameters for 4:2:2 videos.

FIG. 9 shows an example of a video processing apparatus.

FIG. 10 shows a block diagram of a video encoder.

FIG. 11 is a flowchart for an example of a video processing method basedon some implementations of the disclosed technology.

FIG. 12 is a block diagram for an example of a video processing system.

FIG. 13 is a block diagram that illustrates an example video codingsystem.

FIG. 14 is a block diagram that illustrates an encoder in accordancewith some embodiments of the disclosed technology.

FIG. 15 is a block diagram that illustrates a decoder in accordance withsome embodiments of the disclosed technology.

FIGS. 16A and 16B are flowcharts for examples of video processingmethods based on some implementations of the disclosed technology.

DETAILED DESCRIPTION

The present document provides various techniques that can be used by adecoder of image or video bitstreams to improve the quality ofdecompressed or decoded digital video or images. For brevity, the term“video” is used herein to include both a sequence of pictures(traditionally called video) and individual images. Furthermore, a videoencoder may also implement these techniques during the process ofencoding in order to reconstruct decoded frames used for furtherencoding.

Section headings are used in the present document for ease ofunderstanding and do not limit the embodiments and techniques to thecorresponding sections. As such, embodiments from one section can becombined with embodiments from other sections.

1. Brief Summary

This invention is related to video coding technologies. Specifically, itis related cross-component linear model prediction and other codingtools in image/video coding. It may be applied to the existing videocoding standard like HEVC, or the standard (Versatile Video Coding) tobe finalized. It may be also applicable to future video coding standardsor video codec.

2. Video Coding Introduction

Video coding standards have evolved primarily through the development ofthe well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 andH.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the twoorganizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, thevideo coding standards are based on the hybrid video coding structurewherein temporal prediction plus transform coding are utilized. Toexplore the future video coding technologies beyond HEVC, Joint VideoExploration Team (JVET) was founded by VCEG and MPEG jointly in 2015.Since then, many new methods have been adopted by JVET and put into thereference software named Joint Exploration Model (JEM). In April 2018,the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1SC29/WG11 (MPEG) was created to work on the VVC standard targeting at50% bitrate reduction compared to HEVC.

2.1. Color Space and Chroma Subsampling

Color space, also known as the color model (or color system), is anabstract mathematical model which simply describes the range of colorsas tuples of numbers, typically as 3 or 4 values or color components(e.g. RGB). Basically speaking, color space is an elaboration of thecoordinate system and sub-space.

For video compression, the most frequently used color spaces are YCbCrand RGB.

YCbCr, Y′CbCr, or Y Pb/Cb Pr/Cr, also written as YCBCR or Y′CBCR, is afamily of color spaces used as a part of the color image pipeline invideo and digital photography systems. Y′ is the luma component and CBand CR are the blue-difference and red-difference chroma components. Y′(with prime) is distinguished from Y, which is luminance, meaning thatlight intensity is nonlinearly encoded based on gamma corrected RGBprimaries.

Chroma sub sampling is the practice of encoding images by implementingless resolution for chroma information than for luma information, takingadvantage of the human visual system's lower acuity for colordifferences than for luminance.

2.1.1. 4:4:4

Each of the three Y′CbCr components have the same sample rate, thusthere is no chroma subsampling. This scheme is sometimes used inhigh-end film scanners and cinematic post production.

2.1.2. 4:2:2

The two chroma components are sampled at half the sample rate of luma:the horizontal chroma resolution is halved while the vertical chromaresolution is unchanged. This reduces the bandwidth of an uncompressedvideo signal by one-third with little to no visual difference. Anexample of nominal vertical and horizontal locations of 4:2:2 colorformat is depicted in FIG. 1A in VVC working draft.

2.1.3. 4:2:0

In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but asthe Cb and Cr channels are only sampled on each alternate line in thisscheme, the vertical resolution is halved. The data rate is thus thesame. Cb and Cr are each subsampled at a factor of 2 both horizontallyand vertically. There are three variants of 4:2:0 schemes, havingdifferent horizontal and vertical siting.

-   -   In MPEG-2, Cb and Cr are cosited horizontally. Cb and Cr are        sited between pixels in the vertical direction (sited        interstitially).

In JPEG/JFIF, H.261, and MPEG-1, Cb and Cr are sited interstitially,halfway between alternate luma samples.

In 4:2:0 DV, Cb and Cr are co-sited in the horizontal direction. In thevertical direction, they are co-sited on alternating lines.

TABLE 2-1 Sub WidthC and SubHeightC values derived from chroma _format_idc and separate_colour _plane _flag • chroma_format_separate_colour_ Chroma Sub- Sub- id C plane_flag format WidthC HeightC0 0 Monochrome 1 1 1 0 4:2:0 2 2 2 0 4:2:2 2 1 3 0 4:4:4 1 1 3 1 4:4:4 11

2.2. Coding Flow of a Typical Video Codec

FIG. 1B shows an example of encoder block diagram of VVC, which containsthree in-loop filtering blocks: deblocking filter (DF), sample adaptiveoffset (SAO) and ALF. Unlike DF, which uses predefined filters, SAO andALF utilize the original samples of the current picture to reduce themean square errors between the original samples and the reconstructedsamples by adding an offset and by applying a finite impulse response(FIR) filter, respectively, with coded side information signalling theoffsets and filter coefficients. ALF is located at the last processingstage of each picture and can be regarded as a tool trying to catch andfix artifacts created by the previous stages.

2.3. Intra Mode Coding with 67 Intra Prediction Modes

To capture the arbitrary edge directions presented in natural video, thenumber of directional intra modes is extended from 33, as used in HEVC,to 65. The additional directional modes are depicted as red dottedarrows in FIG. 2, and the planar and DC modes remain the same. Thesedenser directional intra prediction modes apply for all block sizes andfor both luma and chroma intra predictions.

Conventional angular intra prediction directions are defined from 45degrees to −135 degrees in clockwise direction as shown in FIG. 2. InVTM, several conventional angular intra prediction modes are adaptivelyreplaced with wide-angle intra prediction modes for the non-squareblocks. The replaced modes are signalled using the original method andremapped to the indexes of wide angular modes after parsing. The totalnumber of intra prediction modes is unchanged, i.e., 67, and the intramode coding is unchanged.

In the HEVC, every intra-coded block has a square shape and the lengthof each of its side is a power of 2. Thus, no division operations arerequired to generate an intra-predictor using DC mode. In VVC, blockscan have a rectangular shape that necessitates the use of a divisionoperation per block in the general case. To avoid division operationsfor DC prediction, only the longer side is used to compute the averagefor non-square blocks.

FIG. 2 shows examples of 67 intra prediction modes.

2.4. Inter Prediction

For each inter-predicted CU, motion parameters consisting of motionvectors, reference picture indices and reference picture list usageindex, and additional information needed for the new coding feature ofVVC to be used for inter-predicted sample generation. The motionparameter can be signalled in an explicit or implicit manner. When a CUis coded with skip mode, the CU is associated with one PU and has nosignificant residual coefficients, no coded motion vector delta orreference picture index. A merge mode is specified whereby the motionparameters for the current CU are obtained from neighbouring CUs,including spatial and temporal candidates, and additional schedulesintroduced in VVC. The merge mode can be applied to any inter-predictedCU, not only for skip mode. The alternative to merge mode is theexplicit transmission of motion parameters, where motion vector,corresponding reference picture index for each reference picture listand reference picture list usage flag and other needed information aresignalled explicitly per each CU.

2.5. Intra Block Copy (IBC)

Intra block copy (IBC) is a tool adopted in HEVC extensions on SCC. Itis well known that it significantly improves the coding efficiency ofscreen content materials. Since IBC mode is implemented as a block levelcoding mode, block matching (BM) is performed at the encoder to find theoptimal block vector (or motion vector) for each CU. Here, a blockvector is used to indicate the displacement from the current block to areference block, which is already reconstructed inside the currentpicture. The luma block vector of an IBC-coded CU is in integerprecision. The chroma block vector rounds to integer precision as well.When combined with AMVR, the IBC mode can switch between 1-pel and 4-pelmotion vector precisions. An IBC-coded CU is treated as the thirdprediction mode other than intra or inter prediction modes. The IBC modeis applicable to the CUs with both width and height smaller than orequal to 64 luma samples.

At the encoder side, hash-based motion estimation is performed for IBC.The encoder performs RD check for blocks with either width or height nolarger than 16 luma samples. For non-merge mode, the block vector searchis performed using hash-based search first. If hash search does notreturn valid candidate, block matching based local search will beperformed.

In the hash-based search, hash key matching (32-bit CRC) between thecurrent block and a reference block is extended to all allowed blocksizes. The hash key calculation for every position in the currentpicture is based on 4×4 sub-blocks. For the current block of a largersize, a hash key is determined to match that of the reference block whenall the hash keys of all 4×4 sub-blocks match the hash keys in thecorresponding reference locations. If hash keys of multiple referenceblocks are found to match that of the current block, the block vectorcosts of each matched reference are calculated and the one with theminimum cost is selected.

In block matching search, the search range is set to cover both theprevious and current CTUs. At CU level, IBC mode is signalled with aflag and it can be signalled as IBC AMVP mode or IBC skip/merge mode asfollows:

-   -   IBC skip/merge mode: a merge candidate index is used to indicate        which of the block vectors in the list from neighbouring        candidate IBC coded blocks is used to predict the current block.        The merge list consists of spatial, HMVP, and pairwise        candidates.    -   IBC AMVP mode: block vector difference is coded in the same way        as a motion vector difference. The block vector prediction        method uses two candidates as predictors, one from left        neighbour and one from above neighbour (if IBC coded). When        either neighbour is not available, a default block vector will        be used as a predictor. A flag is signalled to indicate the        block vector predictor index.

2.6. Palette Mode

For palette mode signalling, the palette mode is coded as a predictionmode for a coding unit, i.e., the prediction modes for a coding unit canbe MODE_INTRA, MODE_INTER, MODE_IBC and MODE_PLT. If the palette mode isutilized, the pixels values in the CU are represented by a small set ofrepresentative colour values. The set is referred to as the palette. Forpixels with values close to the palette colors, the palette indices aresignalled. For pixels with values outside the palette, the pixel isdenoted with an escape symbol and the quantized pixel values aresignalled directly.

To decode a palette encoded block, the decoder needs to decode palettecolors and indices. Palette colors are described by a palette table andencoded by palette table coding tools. An escape flag is signalled foreach CU to indicate if escape symbols are present in the current CU. Ifescape symbols are present, the palette table is augmented by one andthe last index is assigned to the escape mode. Palette indices of allpixels in a CU form a palette index map and are encoded by palette indexmap coding tools.

For coding of the palette table, a palette predictor is maintained. Thepredictor is initialized at the beginning of each slice where predictoris reset to 0. For each entry in the palette predictor, a reuse flag issignalled to indicate whether it is part of the current palette. Thereuse flags are sent using run-length coding of zeros. After this, thenumber of new palette entries are signalled using exponential Golombcode of order 0. Finally, the component values for the new paletteentries are signalled. After encoding the current CU, the palettepredictor will be updated using the current palette, and entries fromthe previous palette predictor which are not reused in the currentpalette will be added at the end of new palette predictor until themaximum size allowed is reached (palette stuffing).

For coding the palette index map, the indices are coded using horizontaland vertical traverse scans as shown in FIG. 3. The scan order isexplicitly signalled in the bitstream using the palette_transpose_flag.

FIG. 3 shows examples of horizontal and vertical traverse scans.

The palette indices are coded using two main palette sample modes:‘INDEX’ and ‘COPY_ABOVE’. The mode is signalled using a flag except forthe top row when horizontal scan is used, the first column when thevertical scan is used, or when the previous mode was ‘COPY_ABOVE’. Inthe ‘COPY_ABOVE’ mode, the palette index of the sample in the row aboveis copied. In the ‘INDEX’ mode, the palette index is explicitlysignalled. For both ‘INDEX’ and ‘COPY_ABOVE’ modes, a run value issignalled which specifies the number pixels that are coded using thesame mode.

The encoding order for index map is as follows: First, the number ofindex values for the CU is signalled. This is followed by signalling ofthe actual index values for the entire CU using truncated binary coding.Both the number of indices as well as the index values are coded inbypass mode. This groups the index-related bypass bins together. Thenthe palette mode (INDEX or COPY_ABOVE) and run are signalled in aninterleaved manner. Finally, the component escape values correspondingto the escape samples for the entire CU are grouped together and codedin bypass mode. An additional syntax element, last_run_type_flag, issignalled after signalling the index values. This syntax element, inconjunction with the number of indices, eliminates the need to signalthe run value corresponding to the last run in the block.

In VTM, dual tree is enabled for I slice which separate the coding unitpartitioning for Luma and Chroma. Hence, in this proposal, palette isapplied on Luma (Y component) and Chroma (Cb and Cr components)separately. If dual tree is disabled, palette will be applied on Y, Cb,Cr components jointly, same as in HEVC palette.

2.7. Cross-Component Linear Model Prediction

A cross-component linear model (CCLM) prediction mode is used in theVVC, for which the chroma samples are predicted based on thereconstructed luma samples of the same CU by using a linear model asfollows:pred_(C)(i,j)=α·rec _(L)′(i,j)+β  (2-1)

where pred_(C)(i,j) represents the predicted chroma samples in a CU andrec_(L)(i,j) represents the downsampled reconstructed luma samples ofthe same CU.

FIG. 4 shows an example of the location of the left and above samplesand the sample of the current block involved in the LM mode.

FIG. 4 shows examples of locations of the samples used for thederivation of α and β.

Besides the above template and left template can be used to calculatethe linear model coefficients together in LM mode, they also can be usedalternatively in the other 2 LM modes, called LM_A, and LM_L modes. InLM_A mode, only the above template is used to calculate the linear modelcoefficients. To get more samples, the above template is extended to(W+H). In LM_L mode, only left template is used to calculate the linearmodel coefficients. To get more samples, the left template is extendedto (H+W). For a non-square block, the above template is extended to W+W,the left template is extended to H+H.

The CCLM parameters (α and β) are derived with at most four neighbouringchroma samples and their corresponding down-sampled luma samples.Suppose the current chroma block dimensions are W×H, then W′ and H′ areset as

-   -   W′=W, H′=H when LM mode is applied;    -   W′=W+H when LM-A mode is applied;    -   H′=H+W when LM-L mode is applied;

The above neighbouring positions are denoted as S[0, −1] . . . S[W′−1,−1] and the left neighbouring positions are denoted as S[−1, 0] . . .S[−1, H′−1]. Then the four samples are selected as

-   -   S[W′/4, −1], S[3 W′/4, −1], S[−1, H′/4], S[−1, 3H′/4] when LM        mode is applied and both above and left neighbouring samples are        available;    -   S[W′/8, −1], S[3 W′/8, −1], S[5 W′/8, −1], S[7 W′/8, −1] when        LM-A mode is applied or only the above neighbouring samples are        available;    -   S[−1, H′/8], S[−1, 3H′/8], S[−1, 5H′/8], S[−1, 7H′/8] when LM-L        mode is applied or only the left neighbouring samples are        available;

The four neighbouring luma samples at the selected positions aredown-sampled and compared four times to find two smaller values: x⁰ _(A)and x¹ _(A), and two larger values: x⁰ _(B) and x¹ _(B). Theircorresponding chroma sample values are denoted as y⁰ _(A), y₁ ^(A), y⁰_(B) and y¹ _(B). Then x_(A), x_(B), y_(A) and y_(B) are derived as:X _(a)(x ⁰ _(A) +x ¹ _(A)+1)>>1;X _(b)=(x ⁰ _(B) +x ¹ _(B)+1)>>1;Y_(a)=(y ⁰ _(A) +y ¹ _(A)+1)>>1;Y _(b)=(y ⁰ _(B) +y ¹ _(B)+1)>>1  (2-2)

Finally, the linear model parameters a and are obtained according to thefollowing equations.

$\begin{matrix}{\alpha = \frac{Y_{a} - Y_{b}}{X_{a} - X_{b}}} & \left( {2\text{-}3} \right) \\{\beta = {Y_{b} - {\alpha \cdot X_{b}}}} & \left( {2\text{-}4} \right)\end{matrix}$

The division operation to calculate parameter α is implemented with alook-up table. To reduce the memory required for storing the table, thediff value (difference between maximum and minimum values) and theparameter α are expressed by an exponential notation. For example, diffis approximated with a 4-bit significant part and an exponent.Consequently, the table for 1/diff is reduced into 16 elements for 16values of the significand as follows:DivTable[ ]=0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0  (2-5)

This would have a benefit of both reducing the complexity of thecalculation as well as the memory size required for storing the neededtables.

To match the chroma sample locations for 4:2:0 video sequences, twotypes of downsampling filter are applied to luma samples to achieve 2 to1 downsampling ratio in both horizontal and vertical directions. Theselection of downsampling filter is specified by a SPS level flag. Thetwo downsampling filters are as follows, which are corresponding to“type-0” and “type-2” content, respectively.

$\begin{matrix}{{{rec}_{L}^{\prime}\left( {i,j} \right)} = {\begin{bmatrix}\begin{matrix}{{{rec}_{L}\left( {{{2i} - 1},{{2j} - 1}} \right)} + {2 \cdot}} \\{{{rec}_{L}\left( {{{2i} - 1},{{2j} - 1}} \right)} +}\end{matrix} \\{{{rec}_{L}\left( {{{2i} + 1},{{2j} - 1}} \right)} + {{rec}_{L}\left( {{{2i} - 1},{2j}} \right)} +} \\{{2 \cdot {{rec}_{L}\left( {{2i},{2j}} \right)}} + {{rec}_{L}\left( {{{2i} + 1},{2j}} \right)} + 4}\end{bmatrix} ⪢ 3}} & \left( {2\text{-}6} \right) \\{{{rec}_{L}^{\prime}\left( {i,j} \right)} = {\begin{bmatrix}{{{rec}_{L}\left( {{2i},{{2j} - 1}} \right)} + {{rec}_{L}\left( {{{2i} - 1},{2j}} \right)} +} \\{{4 \cdot {{rec}_{L}\left( {{2i},{2j}} \right)}} + {{rec}_{L}\left( {{{2i} + 1},{2j}} \right)} +} \\{{{rec}_{L}\left( {{2i},{{2j} + 1}} \right)} + 4}\end{bmatrix} ⪢ 3}} & \left( {2\text{-}7} \right)\end{matrix}$

Note that only one luma line (general line buffer in intra prediction)is used to make the downsampled luma samples when the upper referenceline is at the CTU boundary.

This parameter computation is performed as part of the decoding process,and not just as an encoder search operation. As a result, no syntax isused to convey the α and β values to the decoder.

For chroma intra mode coding, a total of 8 intra modes are allowed forchroma intra mode coding. Those modes include five traditional intramodes and three cross-component linear model modes (LM, LM_A, and LM_L).Chroma mode signalling and derivation process are shown in Table 2-2.Chroma mode coding directly depends on the intra prediction mode of thecorresponding luma block. Since separate block partitioning structurefor luma and chroma components is enabled in I slices, one chroma blockmay correspond to multiple luma blocks. Therefore, for Chroma DM mode,the intra prediction mode of the corresponding luma block covering thecenter position of the current chroma block is directly inherited.

TABLE 2-2 Derivation of chroma prediction mode from luma mode when cclmis enabled Chroma prediction Corresponding luma intra prediction modemode 0 50 18 1 X(0<= X <= 66) 0 66 0 0 0 0 1 50 66 50 50 50 2 18 18 6618 18 3 1 1 1 66 1 4 81 81 81 81 81 5 82 82 82 82 82 6 83 83 83 83 83 70 50 18 1 X

2.8. Block Differential Pulse-Code Modulation Coding (BDPCM)

BDPCM is proposed in JVET-M0057. Due to the shape of the horizontal(resp. vertical) predictors, which use the left (A) (resp. top (B))pixel for prediction of the current pixel, the most throughput-efficientway of processing the block is to process all the pixels of one column(resp. line) in parallel, and to process these columns (resp. lines)sequentially. In order to increase throughput, we introduce thefollowing process: a block of width 4 is divided into two halves with ahorizontal frontier when the predictor chosen on this block is vertical,and a block of height 4 is divided into two halves with a verticalfrontier when the predictor chosen on this block is horizontal.

When a block is divided, samples from one area are not allowed to usepixels from another area to compute the prediction: if this situationoccurs, the prediction pixel is replaced by the reference pixel in theprediction direction. This is shown in FIG. 5 for different positions ofcurrent pixel X in a 4×8 block predicted vertically.

FIG. 5 shows example of dividing a block of 4×8 samples into twoindependently decodable areas.

Thanks to this property, it becomes now possible to process a 4×4 blockin 2 cycles, and a 4×8 or 8×4 block in 4 cycles, and so on, as shown onFIG. 6.

FIG. 6 shows an example order of processing of the rows of pixels tomaximize throughput for 4×N blocks with vertical predictor.

Table 2-3 summarizes the number of cycles required to process the block,depending on the block size. It is trivial to show that any block whichhas both dimensions larger or equal to 8 can be processed in 8 pixelsper cycle or more.

TABLE 2-3 Worst case throughput for blocks of size 4×N, N×4 Block size4×4 4×8, 8×4 4×16, 16×4 4×32, 32×4 Cycles 2 4 8 16 Pixels 16 32 64 128Throughput 8 8 8 8 (pixels/cycle)

2.9. Quantized Residual Domain BDPCM

In JVET-N0413, quantized residual domain BDPCM (denote as RBDPCMhereinafter) is proposed. The intra prediction is done on the entireblock by sample copying in prediction direction (horizontal or verticalprediction) similar to intra prediction. The residual is quantized andthe delta between the quantized residual and its predictor (horizontalor vertical) quantized value is coded.

For a block of size M (rows)×N (cols), let r_(i,j), 0≤i≤M−1, 0≤j≤N−1 bethe prediction residual after performing intra prediction horizontally(copying left neighbour pixel value across the predicted block line byline) or vertically (copying top neighbour line to each line in thepredicted block) using unfiltered samples from above or left blockboundary samples. Let Q(r_(i,j)), 0≤i≤M−1, 0≤j≤N−1 denote the quantizedversion of the residual r_(i,j), where residual is difference betweenoriginal block and the predicted block values. Then the block DPCM isapplied to the quantized residual samples, resulting in modified M×Narray {tilde over (R)} with elements {tilde over (r)}_(i,j). Whenvertical BDPCM is signalled:

$\begin{matrix}{{\overset{\sim}{r}}_{i,j} = \left\{ {\begin{matrix}{{{Q\left( r_{i,j} \right)},}\ } & {{i = 0},{0 \leq j \leq \left( {N - 1} \right)}} \\{{{Q\left( r_{i,j} \right)} - {Q\left( r_{{({i - 1})},j} \right)}},} & {{1 \leq i \leq \left( {M - 1} \right)},{0 \leq j \leq \left( {N - 1} \right)}}\end{matrix}.} \right.} & \left( {2\text{-}8} \right)\end{matrix}$

For horizontal prediction, similar rules apply, and the residualquantized samples are obtained by

$\begin{matrix}{{\overset{\sim}{r}}_{i,j} = \left\{ {\begin{matrix}{{{Q\left( r_{i,j} \right)},}\ } & {{0 \leq i \leq \left( {M - 1} \right)},{j = 0}} \\{{{Q\left( r_{i,j} \right)} - {Q\left( r_{i,{({j - 1})}} \right)}},} & {{0 \leq i \leq \left( {M - 1} \right)},{1 \leq j \leq \left( {N - 1} \right)}}\end{matrix}.} \right.} & \left( {2\text{-}9} \right)\end{matrix}$

The residual quantized samples {tilde over (r)}_(i,j) are sent to thedecoder.

On the decoder side, the above calculations are reversed to produceQ(r_(i,j)), 0≤i≤M−1, 0≤j≤N−1. For vertical prediction case,Q(r _(i,j))=Σ_(k=0) ^(i) {tilde over (r)}_(k,j),0≤i≤(M−1),0≤j≤(N−1).  (2-10)

For horizontal case,Q(r _(i,j))=Σ_(k=0) ^(j) {tilde over (r)}_(i,j),0≤i≤(M−1),0≤j≤(N−1).  (2-11)

The inverse quantized residuals, Q⁻¹ (Q(r_(i,j))), are added to theintra block prediction values to produce the reconstructed samplevalues.

The main benefit of this scheme is that the invert DPCM can be done onthe fly during coefficient parsing simply adding the predictor as thecoefficients are parsed or it can be performed after parsing.

Transform skip is always used in quantized residual domain BDPCM.

2.10. Multiple Transform Set (MTS) in VVC

In VTM, large block-size transforms, up to 64×64 in size, are enabled,which is primarily useful for higher resolution video, e.g., 1080p and4K sequences. High frequency transform coefficients are zeroed out forthe transform blocks with size (width or height, or both width andheight) equal to 64, so that only the lower-frequency coefficients areretained. For example, for an M×N transform block, with M as the blockwidth and N as the block height, when M is equal to 64, only the left 32columns of transform coefficients are kept. Similarly, when N is equalto 64, only the top 32 rows of transform coefficients are kept. Whentransform skip mode is used for a large block, the entire block is usedwithout zeroing out any values. The VTM also supports configurable maxtransform size in SPS, such that encoder has the flexibility to chooseup to 16-length, 32-length or 64-length transform size depending on theneed of specific implementation.

In addition to DCT-II which has been employed in HEVC, a MultipleTransform Selection (MTS) scheme is used for residual coding both interand intra coded blocks. It uses multiple selected transforms from theDCT8/DST7. The newly introduced transform matrices are DST-VII andDCT-VIII. Table 2-4 shows the basis functions of the selected DST/DCT.

TABLE 2-4 Transform basis functions of DCT-II/VIII and DSTVII forN-point input Transform Type Basis function T_(i)(j), i, j = 0, 1, . . ., N − 1 DCT-II $\begin{matrix}{{T_{i}(j)} = {\omega_{0} \cdot \sqrt{\frac{2}{N}} \cdot {\cos\left( \frac{\pi \cdot i \cdot \left( {{2j} + 1} \right)}{2N} \right)}}} \\{{where},{\omega_{0} = \left\{ \begin{matrix}\sqrt{\frac{2}{N}} & {i = 0} \\1 & {i \neq 0}\end{matrix} \right.}}\end{matrix}$ DCT-VIII${T_{i}(j)} = {\sqrt{\frac{4}{{2N} + 1}} \cdot {\cos\left( \frac{\pi \cdot \left( {{2i} + 1} \right) \cdot \left( {{2j} + 1} \right)}{{4N} + 2} \right)}}$DST-VII${T_{i}(j)} = {\sqrt{\frac{4}{{2N} + 1}} \cdot {\sin\left( \frac{\pi \cdot \left( {{2i} + 1} \right) \cdot \left( {j + 1} \right)}{{2N} + 1} \right)}}$

In order to keep the orthogonality of the transform matrix, thetransform matrices are quantized more accurately than the transformmatrices in HEVC. To keep the intermediate values of the transformedcoefficients within the 16-bit range, after horizontal and aftervertical transform, all the coefficients are to have 10-bit.

In order to control MTS scheme, separate enabling flags are specified atSPS level for intra and inter, respectively. When MTS is enabled at SPS,a CU level flag is signalled to indicate whether MTS is applied or not.Here, MTS is applied only for luma. The MTS CU level flag is signalledwhen the following conditions are satisfied.

-   -   Both width and height smaller than or equal to 32    -   CBF flag is equal to one

If MTS CU flag is equal to zero, then DCT2 is applied in bothdirections. However, if MTS CU flag is equal to one, then two otherflags are additionally signalled to indicate the transform type for thehorizontal and vertical directions, respectively. Transform andsignalling mapping table as shown in Table 2-5. Unified the transformselection for ISP and implicit MTS is used by removing the intra-modeand block-shape dependencies. If current block is ISP mode or if thecurrent block is intra block and both intra and inter explicit MTS ison, then only DST7 is used for both horizontal and vertical transformcores. When it comes to transform matrix precision, 8-bit primarytransform cores are used. Therefore, all the transform cores used inHEVC are kept as the same, including 4-point DCT-2 and DST-7, 8-point,16-point and 32-point DCT-2. Also, other transform cores including64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7 andDCT-8, use 8-bit primary transform cores.

TABLE 2-5 Transform and signalling mapping table MTS_CU_ MTS_Hor_MTS_Ver_ flag flag flag Intra/inter Horizontal Vertical 0 DCT2 1 0 0DST7 DST7 0 1 DCT8 DST7 1 0 DST7 DCT8 1 1 DCT8 DCT8

To reduce the complexity of large size DST-7 and DCT-8, High frequencytransform coefficients are zeroed out for the DST-7 and DCT-8 blockswith size (width or height, or both width and height) equal to 32. Onlythe coefficients within the 16×16 lower-frequency region are retained.

As in HEVC, the residual of a block can be coded with transform skipmode. To avoid the redundancy of syntax coding, the transform skip flagis not signalled when the CU level MTS_CU_flag is not equal to zero. Theblock size limitation for transform skip is the same to that for MTS inJEM4, which indicate that transform skip is applicable for a CU whenboth block width and height are equal to or less than 32. Note thatimplicit MTS transform is set to DCT2 when LFNST or MIP is activated forthe current CU. Also, the implicit MTS can be still enabled when MTS isenabled for inter coded blocks.

2.11. Low-Frequency Non-Separable Transform (LFNST)

In VVC, LFNST (low-frequency non-separable transform), which is known asreduced secondary transform, is applied between forward primarytransform and quantization (at encoder) and between de-quantization andinverse primary transform (at decoder side) as shown in FIG. 7. InLFNST, 4×4 non-separable transform or 8×8 non-separable transform isapplied according to block size. For example, 4×4 LFNST is applied forsmall blocks (i.e., min (width, height)<8) and 8×8 LFNST is applied forlarger blocks (i.e., min (width, height)>4).

FIG. 7 shows an example of a low-Frequency Non-Separable Transform(LFNST) process.

Application of a non-separable transform, which is being used in LFNST,is described as follows using input as an example. To apply 4×4 LFNST,the 4×4 input block X

$\begin{matrix}{X = \begin{bmatrix}X_{00} & X_{01} & X_{02} & X_{03} \\X_{10} & X_{11} & X_{12} & X_{13} \\X_{20} & X_{21} & X_{22} & X_{23} \\X_{30} & X_{31} & X_{32} & X_{33}\end{bmatrix}} & \left( {2\text{-}12} \right)\end{matrix}$

is first represented as a vector

:

=[X ₀₀ X ₀₁ X ₀₂ X ₀₃ X ₁₀ X ₁₁ X ₁₂ X ₁₃ X ₂₀ X ₂₁ X ₂₂ X ₂₃ X ₃₀ X ₃₁X ₃₂ X ₃₃]^(T)   (2-13)

The non-separable transform is calculated as

=T·

, where

indicates the transform coefficient vector, and Tis a 16×16 transformmatrix. The 16×1 coefficient vector

is subsequently re-organized as 4×4 block using the scanning order forthat block (horizontal, vertical or diagonal). The coefficients withsmaller index will be placed with the smaller scanning index in the 4×4coefficient block.

2.11.1. Reduced Non-Separable Transform

LFNST (low-frequency non-separable transform) is based on direct matrixmultiplication approach to apply non-separable transform so that it isimplemented in a single pass without multiple iterations. However, thenon-separable transform matrix dimension needs to be reduced to minimizecomputational complexity and memory space to store the transformcoefficients. Hence, reduced non-separable transform (or RST) method isused in LFNST. The main idea of the reduced non-separable transform isto map an N (N is commonly equal to 64 for 8×8 NSST) dimensional vectorto an R dimensional vector in a different space, where N/R (R<N) is thereduction factor. Hence, instead of N×N matrix, RST matrix becomes anR×N matrix as follows:

$\begin{matrix}{T_{R \times N} = \begin{bmatrix}t_{11} & t_{12} & t_{13} & {\;\ldots} & t_{1N} \\t_{21} & t_{22} & t_{23} & \; & t_{2N} \\\; & \vdots & \; & \ddots & \vdots \\t_{R\; 1} & t_{R\; 2} & t_{R\; 3} & \cdots & t_{RN}\end{bmatrix}} & \left( {2\text{-}14} \right)\end{matrix}$

where the R rows of the transform are R bases of the N dimensionalspace. The inverse transform matrix for RT is the transpose of itsforward transform. For 8×8 LFNST, a reduction factor of 4 is applied,and 64×64 direct matrix, which is conventional 8×8 non-separabletransform matrix size, is reduced to 16×48 direct matrix. Hence, the48×16 inverse RST matrix is used at the decoder side to generate core(primary) transform coefficients in 8×8 top-left regions. When 16×48matrices are applied instead of 16×64 with the same transform setconfiguration, each of which takes 48 input data from three 4×4 blocksin a top-left 8×8 block excluding right-bottom 4×4 block. With the helpof the reduced dimension, memory usage for storing all LFNST matrices isreduced from 10 KB to 8 KB with reasonable performance drop. In order toreduce complexity LFNST is restricted to be applicable only if allcoefficients outside the first coefficient sub-group arenon-significant. Hence, all primary-only transform coefficients have tobe zero when LFNST is applied. This allows a conditioning of the LFNSTindex signalling on the last-significant position, and hence avoids theextra coefficient scanning in the current LFNST design, which is neededfor checking for significant coefficients at specific positions only.The worst-case handling of LFNST (in terms of multiplications per pixel)restricts the non-separable transforms for 4×4 and 8×8 blocks to 8×16and 8×48 transforms, respectively. In those cases, the last-significantscan position has to be less than 8 when LFNST is applied, for othersizes less than 16. For blocks with a shape of 4×N and N×4 and N>8, theproposed restriction implies that the LFNST is now applied only once,and that to the top-left 4×4 region only. As all primary-onlycoefficients are zero when LFNST is applied, the number of operationsneeded for the primary transforms is reduced in such cases. From encoderperspective, the quantization of coefficients is remarkably simplifiedwhen LFNST transforms are tested. A rate-distortion optimizedquantization has to be done at maximum for the first 16 coefficients (inscan order), the remaining coefficients are enforced to be zero.

2.11.2. LFNST Transform Selection

There are totally 4 transform sets and 2 non-separable transformmatrices (kernels) per transform set are used in LFNST. The mapping fromthe intra prediction mode to the transform set is pre-defined as shownin Table 2-6. If one of three CCLM modes (INTRA_LT_CCLM, INTRA_T_CCLM orINTRA_L_CCLM) is used for the current block (81<=predModeIntra 83),transform set 0 is selected for the current chroma block. For eachtransform set, the selected non-separable secondary transform candidateis further specified by the explicitly signalled LFNST index. The indexis signalled in a bit-stream once per Intra CU after transformcoefficients.

TABLE 2-6 Transform selection table Tr. set IntraPredMode indexIntraPredMode<0 1 0 <=IntraPredMode <=1 0 2 <=IntraPredMode <=12 1 13<=IntraPredMode <=23 2 24 <=IntraPredMode <=44 3 45 <=IntraPredMode <=552 56 <=IntraPredMode<= 80 1 81 <=IntraPredMode<= 83 0

2.11.3. LFNST Index Signalling and Interaction with Other Tools

Since LFNST is restricted to be applicable only if all coefficientsoutside the first coefficient sub-group are non-significant, LFNST indexcoding depends on the position of the last significant coefficient. Inaddition, the LFNST index is context coded but does not depend on intraprediction mode, and only the first bin is context coded. Furthermore,LFNST is applied for intra CU in both intra and inter slices, and forboth Luma and Chroma. If a dual tree is enabled, LFNST indices for Lumaand Chroma are signalled separately. For inter slice (the dual tree isdisabled), a single LFNST index is signalled and used for both Luma andChroma.

When ISP mode is selected, LFNST is disabled and RST index is notsignalled, because performance improvement was marginal even if RST isapplied to every feasible partition block. Furthermore, disabling RSTfor ISP-predicted residual could reduce encoding complexity. LFNST isalso disabled and the index is not signalled when MIP mode is selected.

Considering that a large CU greater than 64×64 is implicitly split (TUtiling) due to the existing maximum transform size restriction (64×64),an LFNST index search could increase data buffering by four times for acertain number of decode pipeline stages. Therefore, the maximum sizethat LFNST is allowed is restricted to 64×64. Note that LFNST is enabledwith DCT2 only.

2.12. Transform Skip for Chroma

Chroma transform skip (TS) is introduced in VVC. The motivation is tounify TS and MTS signalling between luma and chroma by relocatingtransform_skip_flag and mts_idx into residual_coding part. One contextmodel is added for chroma TS. No context model and no binarization arechanged for the mts_idx. In addition, TS residual_coding is also appliedwhen chroma TS is used.

Semantics

transform_skip_flag[x0][y0][cIdx] specifies whether a transform isapplied to the associated transform block or not. The array indices x0,y0 specify the location (x0, y0) of the top-left luma sample of theconsidered transform block relative to the top-left luma sample of thepicture. transform_skip_flag[x0][y0][cIdx] equal to 1 specifies that notransform is applied to the current transform block. The array indexcIdx specifies an indicator for the colour component; it is equal to 0for luma, equal to 1 for Cb and equal to 2 for Cr.transform_skip_flag[x0][y0][cIdx] equal to 0 specifies that the decisionwhether transform is applied to the current transform block or notdepends on other syntax elements. When transform_skip_flag[x0][y0][cIdx]is not present, it is inferred to be equal to 0.

2.13. BDPCM for Chroma

In addition to chroma TS support, BDPCM is added to chroma components.If sps_bdpcm_enable_flag is 1, a further syntax elementsps_bdpcm_chroma_enable_flag is added to the SPS. The flags have thefollowing behaviour, as indicated in Table 2-7.

TABLE 2-7 sps flags for luma and chroma BDPCM sps_bdpcm_ sps_bdpcm_enable _flag chroma_enable _flag behaviour 0 not written BPDCM is notused in the sequence 1 0 BDPCM is available for luma only 1 1 BDPCM isavailable for luma and chroma

When BDPCM is available for luma only, the current behaviour isunchanged. When BDPCM is also available for chroma, a bdpcm_chroma_flagis sent for each chroma block. This indicates whether BDPCM is used onthe chroma blocks. When it is on, BDPCM is used for both chromacomponents, and an additional bdpcm_dir_chroma flag is coded, indicatingthe prediction direction used for both chroma components.

The deblocking filter is de-activated on a border between two Block-DPCMblocks, since neither of the blocks uses the transform stage usuallyresponsible for blocking artifacts. This deactivation happensindependently for luma and chroma components.

3. Examples of Technical Problems Solved by the Disclosed Solutions

The current design of derivation of linear parameters in CCLM and TS hasthe following problems:

-   -   1. For the non-4:4:4 color format, the derivation of linear        parameters in CCLM involves neighbouring chroma samples and        down-sampled collocated neighbouring luma samples. As shown in        FIG. 8, in current VVC, when the nearest line is not at the CTU        boundary, the downsampled collocated neighbouring top luma        samples are derived using the second line above current block        for 4:2:2 videos. However, for the 4:2:2 videos, the vertical        resolution is unchanged. Therefore, there is phase shift between        the downsampled collocated neighbouring top luma samples and the        neighbouring chroma samples.

FIG. 8 shows an example of neighbouring chroma samples and downsampledcollocated neighbouring luma samples used in the derivation of CCLMparameters for 4:2:2 videos.

-   -   2. In current VVC, the same maximum block size is used in the        condition check for signalling of luma transform_skip_flag and        signalling of chroma transform_skip_flag. Such a design doesn't        take the color format into consideration which is not desirable.        -   a. similar problem also exists for signalling of luma BDPCM            flag and signalling of chroma BDPCM flag wherein the same            maximum block size is used in the condition check.

4. A Listing of Embodiments and Techniques

The listing below should be considered as examples to explain generalconcepts. These items should not be interpreted in a narrow way.Furthermore, these items can be combined in any manner.

In this document, the term CCLM′ represents a coding tool that utilizescross-color component information to predict samples/residuals forcurrent color component or to derive reconstruction of samples incurrent color component. It is not limited to the CCLM technologiesdescribed in VVC.

Derivation of Linear Parameters in CCLM

-   -   1. When deriving the CCLM parameters for a chroma block, one or        multiple above neighboring lines of its collocated luma block        may be used to derive its downsampled collocated neighbouring        top luma samples.        -   a. In one example, when the current chroma block is not at            the top CTU boundary, the nearest above line of the            collocated luma block, instead of the second line above, may            be used for derivation of the downsampled collocated            neighbouring top luma samples.            -   i. In one example, one same downsampling filter may be                used for deriving the downsampled collocated                neighbouring top luma samples and the downsampled                collocated neighbouring left luma samples.                -   1) For example, [1 2 1] filter may be used. More                    specifically,                    pDsY[x]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2,                    wherein pY[2*x][−1], pY[2*x−1][−1], pY[2*x+1][−1]                    are luma samples from the nearest above neighboring                    line and pDstY[x] represents the downsampled                    collocated neighbouring top luma samples.            -   ii. In one example, different downsampling filters                (e.g., different filter taps/different filter                coefficients) may be used for deriving the downsampled                collocated neighbouring top luma samples and the                downsampled collocated neighbouring left luma samples.            -   iii. In one example, one same downsampling filter may be                used for deriving the downsampled collocated                neighbouring top luma samples regardless of the position                of the chroma block (e.g., the chroma block may be or                may not be at a top CTU boundary).            -   iv. In one example, the above methods may be only                applied to images/videos in 4:2:2 format.        -   b. In one example, when the current chroma block is not at            the top CTU boundary, above neighbouring luma samples,            including the nearest above line of the collocated luma            block, but excluding the second line above, may be used for            derivation of the downsampled collocated neighbouring top            luma samples.        -   c. In one example, the derivation of the downsampled            collocated neighbouring top luma samples may depend on            samples located at multiple lines.            -   i. In one example, it may depend on both the second                nearest line and the nearest line above the collocated                luma block.            -   ii. In one example, the downsampled collocated                neighbouring top luma samples may be derived using one                same downsampling filter for different colour formats                (e.g. 4:2:0 and 4:2:2).                -   1) In one example, 6-tap filter (e.g., [1 2 1; 1 2                    1]) may be utilized.                -    a) In one example, the downsampled collocated                    neighbouring top luma samples may be derived as:                    pDsY[x]=(pY[2*x−1][−2]+2*pY[2*x][−2]+pY[2*x+1][−2]+pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+4)>>3                    wherein pY are corresponding luma samples and                    pDstY[x] represents the downsampled collocated                    neighbouring top luma samples.                -    b) Alternatively, furthermore, the above method may                    be applied when sps_cclm_colocated_chroma_flag is                    equal to 0.                -   2) In one example, 5-tap filter (e.g., [0 1 0; 1 4                    1; 0 1 0]) may be utilized.                -    a) In one example, the downsampled collocated                    neighbouring top luma samples may be derived as:                    pDsY[x]=(pY[2*x][−2]+pY[2*x−1][−1]+4*pY[2*x][−1]+pY[2*x+1][−1]+pY[2*x][0]+4)>>3                    wherein pY are corresponding luma samples and                    pDstY[x] represents the downsampled collocated                    neighbouring top luma samples.                -    b) Alternatively, furthermore, the above method may                    be applied when sps_cclm_colocated_chroma_flag is                    equal to 1.            -   iii. In one example, the above methods may be only                applied to images/videos in 4:2:2 format.                Maximum block sizes of transform skip coded blocks                (e.g., with transform_skip_flag equal to 1, or BDPCM or                other modes that bypass transform process/use identity                transform)    -   2. Maximum block size of transform skip coded blocks may be        dependent on the colour component. Denote the maximum block size        of transform skip coded blocks for luma and chroma by MaxTsSizeY        and MaxTsSizeC, respectively.        -   a. In one example, maximum block sizes for luma and chroma            components may be different.        -   b. In one example, maximum block sizes for two chroma            components may be different.        -   c. In one example, maximum block sizes for luma and chroma            components or for each colour component may be signalled            separately.            -   i. In one example, MaxTsSizeC/MaxTsSizeY may be                signalled at sequence level/picture level/slice                level/tile group level, such as in sequence                header/picture header/SPS/VPS/DPS/PPS/APS/slice                header/tile group header.            -   ii. In one example, the MaxTsSizeY may be conditionally                signalled, such as according to transform skip is                enabled or not/BDPCM is enabled or not.            -   iii. In one example, the MaxTsSizeC may be conditionally                signalled, such as according to colour format/transform                skip is enabled or not/BDPCM is enabled or not.            -   iv. Alternatively, predictive coding between maximum                block sizes for luma and chroma components may be                utilized.        -   d. In one example, MaxTsSizeC may depend on MaxTsSizeY.            -   i. In one example, MaxTsSizeC may be set equal to                MaxTsSizeY.            -   ii. In one example, MaxTsSizeC may be set equal to                MaxTsSizeY/N (N is an integer). For example, N=2.        -   e. In one example, MaxTsSizeC may be set according to the            chroma subsampling ratios.            -   i. In one example, MaxTsSizeC is set equal to                MaxTsSizeY>>SubWidthC, wherein SubWidthC is defined in                Table 2-1.            -   ii. In one example, MaxTsSizeC is set equal to                MaxTsSizeY>>SubHeightC, wherein SubHeightC is defined in                Table 2-1.            -   iii. In one example, MaxTsSizeC is set equal to                MaxTsSizeY>>max (SubWidthC, SubHeightC).            -   iv. In one example, MaxTsSizeC is set equal to                MaxTsSizeY>>min (SubWidthC, SubHeightC).    -   3. Maximum allowed block size width and height for a transform        coded block may be defined differently.        -   a. In one example, the maximum allowed block size width and            height may be signalled separately.        -   b. In one example, the maximum allowed block size width and            height for a chroma transform coded block may be denoted as            MaxTsSizeWC and MaxTsSizeHC, respectively. MaxTsSizeWC may            be set equal to MaxTsSizeY>>SubWidthC and MaxTsSizeHC may be            set equal to MaxTsSizeY>>SubHeightC.            -   i. In one example, the MaxTsSizeY is the one defined in                bullet 2.    -   4. Whether to signal a transform_skip_flag for a chroma block        (e.g., transform_skip_flag[x0][y0][1] and/or        transform_skip_flag[x0][y0][2]) may depend on the maximum        allowed size for chroma transform skip coded blocks.        -   a. In one example, the chroma transform_skip_flag may be            conditionally signalled according to the following            conditions:            -   i. In one example, the conditions are: tbW is less than                or equal to MaxTsSizeC and tbH is less than or equal to                MaxTsSizeC, wherein tbW and tbH are the width and height                of the current chroma block.                -   1) In one example, MaxTsSizeC may be defined as that                    in bullets 2-3.            -   ii. In one example, the conditions are: tbW is less than                or equal to MaxTsSizeWC and tbH is less than or equal to                MaxTsSizeHC, wherein tbW and tbH are the width and                height of the current chroma block, MaxTsSizeWC and                MaxTsSizeHC represent the maximum allowed block size                width and height, respectively, for chroma transform                skip coded blocks.                -   1) In one example, MaxTsSizeWC and/or MaxTsSizeHC                    may be defined as that in bullet 3.        -   b. In one example, the above methods may be applicable to            the coding of chroma BDPCM flags (e.g.,            intra_bdpcm_chroma_flag) by replacing ‘transform skip’ by            ‘BDPCM’.    -   5. Instead of coding two TS flags for two chroma color        component, it is proposed to use one syntax to indicate the        usage of TS for the two chroma color components.        -   a. In one example, instead of coding            transform_skip_flag[x0][y0][1] and/or            transform_skip_flag[x0][y0][2]), a single syntax element            (e.g., TS_chroma_flag) may be coded.            -   i. In one example, the value of the single syntax                element is a binary value.                -   1) Alternatively, furthermore, the two chroma                    component blocks share the same on/off control of TS                    mode according to the single syntax element.                -    a) In one example, the value of the single syntax                    element equal to 0 indicates TS is disabled for                    both.                -    b) In one example, the value of the single syntax                    element equal to 0 indicates TS is enabled for both.                -   2) Alternatively, furthermore, a second syntax                    element may be further singled based on whether the                    value of the single syntax element is equal to K                    (e.g., K=1).                -    a) In one example, the value of the single syntax                    element equal to 0 indicates TS is disabled for                    both; the value of the single syntax element equal                    to 0 indicates TS is enabled for at least one of the                    two chroma component.                -    b) The second syntax element may be used to                    indicate TS is applied to which one of the two                    chroma components and/or TS is applied to both of                    them.            -   ii. In one example, the value of the single syntax                element is a non-binary value.                -   1) In one example, the value of the single syntax                    element equal to K0 indicates TS is disabled for                    both                -   2) In one example, the value of the single syntax                    element equal to K1 indicates TS is enabled for the                    first chroma color component and disabled for the                    second color component.                -   3) In one example, the value of the single syntax                    element equal to K2 indicates TS is disabled for the                    first chroma color component and enabled for the                    second color component.                -   4) In one example, the value of the single syntax                    element equal to K3 indicates TS is enabled for                    both.                -   5) In one example, the single syntax element may be                    coded with fixed length, unary, truncated unary,                    k-th order EG binarization methods.            -   iii. In one example, the single syntax element and/or                second syntax element may be context coded or bypass                coded.                General Features    -   6. Whether to and/or how to apply the disclosed methods above        may be signalled at sequence level/picture level/slice        level/tile group level, such as in sequence header/picture        header/SPS/VPS/DPS/PPS/APS/slice header/tile group header.    -   7. Whether to and/or how to apply the disclosed methods above        may be dependent on coded information, such as color format,        single/dual tree partitioning.

5. Embodiments

This section shows example embodiments and ways to modify the currentVVC standard to describe these embodiments. The changes to the VVCspecification are highlighted in bold and Italic. Deleted texts aremarked with double brackets (e.g., [[a]] denotes the deletion of thecharacter “a”).

5.1. Embodiment 1

The working draft specified in WET-P2001-v9 may be changed as below.

8.4.5.2.13 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLMIntra Prediction Mode

3. The down-sampled collocated luma samples pDsY[x][y] with x=0 . . .nTbW−1, y=0 . . . nTbH−1 are derived as follows:

-   -   If both SubWidthC and SubHeightC are equal to 1, the following        applies:        -   pDsY[x][y] with x=1 . . . nTbW−1, y=1 . . . nTbH−1 is            derived as follows:            pDstY[x][y]=pY[x][y]  (8-159)    -   Otherwise, the following applies:        -   The one-dimensional filter coefficients array F1 and F2, and            the 2-dimensional filter coefficients arrays F3 and F4 are            specified as follows.            F1=[i]=1, with i=0 . . . 1  (8-160)            F2[0]=1,F2[1]=2,F2[2]=1  (8-161)            F3[i][j]=F4[i][j]=0, with i=0 . . . 2,j=0 . . . 2  (8-162)            -   If both SubWidthC and SubHeightC are equal to 2, the                following applies:                F1[0]=1,F1[1]=1  (8-163)                F3[0][1]=1,F3[1][1]=4,F3[2][1]=1,F3[1][0]=1,F3[1][2]=1  (8-164)                F4[0][1]=1,F4[1][1]=2,F4[2][1]=1  (8-165)                F4[0][2]=1,F4[1][2]=2,F4[2][2]=1  (8-166)            -   Otherwise, the following applies:                F1[0]=2,F1[1]=0  (8-167)                F3[1][1]=8  (8-168)                F4[0][1]=2,F4[1][1]=4,F4[2][1]=2,  (8-169)                5. When numSampT is greater than 0, the selected                neighbouring top chroma samples pSelC[idx] are set equal                to p[pickPosT[idx−cntL]][−1] with idx=cntL . . .                cntL+cntT−1, and the down-sampled neighbouring top luma                samples pSelDsY[idx] with idx=0 . . . cntL+cntT−1 are                specified as follows:    -   Otherwise (sps_cclm_colocated_chroma_flag is equal to 0), the        following applies:        -   If x is greater than 0, the following applies:            -   If bCTUboundary is equal to FALSE, the following                applies:                pSelDsY[idx]=(F4[0][1]*pY[SubWidthC*x−1][[[−2]]−1]+F4[0][2]*pY[SubWidthC*x−1][[[−1]]−2]+F4[1][1]*pY[SubWidthC*x][[[−2]]−1]+F4[1][2]*pY[SubWidthC*x][[[−1]]−2]+F4[2][1]*pY[SubWidthC*x+1][[[−2]]−1]+F4[2][2]*pY[SubWidthC*x+1][[[−1]]−2]+4)>>3  (8-193)            -   Otherwise (bCTUboundary is equal to TRUE), the following                applies:                pSelDsY[idx]=(F2[0]*pY[SubWidthC*x−1][−1]+F2[1]*pY[SubWidthC*x][−1]+F2[2]*pY[SubWidthC*x+1][−1]+2)>>2  (8-194)        -   Otherwise (x is equal to 0), the following applies:            -   If availTL is equal to TRUE and bCTUboundary is equal to                FALSE, the following applies:                pSelDsY[idx]=(F4[0][1]*pY[−1][[[−2]]−1]+F4[0][2]*pY[−1][[[−1]]−2]+F4[1][1]*pY[0][[[−2]]−1]+F4[1][2]*pY[0][[[−1]]−2]+F4[2][1]*pY[1][[[−2]]−1]+F4[2][2]*pY[1][[[−1]]−2]+4)>>3  (8-195)            -   Otherwise, if availTL is equal to TRUE and bCTUboundary                is equal to TRUE, the following applies:                pSelDsY[idx]=(F2[0]*pY[−1][−1]+F2[1]*pY[0][−1]+F2[2]*pY[1][−1]+2)>>2  (8-196)            -   Otherwise, if availTL is equal to FALSE and bCTUboundary                is equal to FALSE, the following applies:                pSelDsY[idx]=(F1[1]*pY[0][−2]+F1[0]*pY[0][−1]+1)>>1  (8-197)            -   Otherwise (availTL is equal to FALSE and bCTUboundary is                equal to TRUE), the following applies:                pSelDsY[idx]=pY[0][−1]  (8-198)

5.2. Embodiment 2

This embodiment shows an example on chroma transform_skip_flag codingaccording to maximum allowed transform skip coded block sizes. Theworking draft specified in JVET-P2001-v9 may be changed as below.

7.3.9.10 Transform Unit Syntax

 if( tu_cbf_luma[x0] [y 0] && treeType != DUAL_TREE_CHROMA) {   if(sps_transform_skip_enabled_flag && !BdpcmFlag [x0][y0] [0] &&..   tbWidth <= MaxTsSize && tbHeight <= MaxTsSize &&    ( IntraSubPartitionsSplit[x0] [y]= = ISP_NO_ SPLIT) && !cu_sbt_flag)   transform_skip_flag[x0][y0][0 ] ae(v)   if( !transform_skip_flagx0][y0][0])    residual_coding( x0, y0, Log2(tbWidth), Log2(tbHeight),0)   else    residual_ts_coding( x0, y0, Log2(tbWidth), Log2(tbHeight ),0)  }  if( tu_cbf_cb[x0] [y0] && treeType != DUAL_TREE_LUMA)   if(sps_transform_skip_enabled_flag && !BdpcmFlag [x0][y0][1] &&    wC <=(MaxTsSize >>

 && hC <= (MaxTsSize >>

 && !cu_sbt_flag)    transform skip_flag[x][y0][1] ae(v)   if(!transform_skip_flag[x0][y0][1])    residual_coding( xC, yC, Log2(wC),Log2(hC), 1)   else    residual_ts_coding( xC,yC, Log2(wC), Log2(hC), 1) if( tu_cbf_cr[x0][y0] && treeType != DUAL_TREE_LUMA &&   !(tu_cbf_cb[x0][y0] && tu_joint_cbcr_residual_flag[x0][y0])) {   if(sps_transform_skip_enabled_flag && !BdpcmFlag [x0] [y0][2] &&    wC <=(MaxTsSize >>

 && hC <= (Ma xTsSize >>

) && !cu_sbt_flag)    transform skip_flag[x0][y0][2] ae(v)   if(!transform_skip_flag [x0] [y0][2] )    residual_coding( xC, yC,Log2(wC), Log2(hC), 2)   else    residual_ts_coding( xC,yC, Log2(wC),Log2(hC), 2)  }

5.3. Embodiment 3

This embodiment shows an example on chroma BDPCM flag coding accordingto maximum allowed chroma transform skip coded block sizes. The workingdraft specified in JVET-P2001-v9 may be changed as below.

7.3.9.5 Coding Unit Syntax

 if( ( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA) &&   ChromaArrayType != 0) {   if( pred_mode_plt_flag && treeType = =DUAL_TREE_CHROMA)    palette_coding( x0, y 0, cbWidth /

, cb Height /

 1, 2 )   else {    if( !cu_act_enabled_flag) {     if( cbWidth <=(MaxTsSize >>

) && cbHeight <= (MaxTsSize >>

) &&     sps_bdpcm_chroma_enabled_flag) {     intra_bdpcm chroma_flagae(v)     if( intra_bdpcm_chroma_flag)    intra_bdpcm_chroma_dir_flagae(v)     } else {      if( Cclm Enabled)       cclm_mode_flag ae(v)     if( cclm_mode_flag)       cclm_mode_idx ae(v)      else      intra_chroma_pred_mode ae(v)     }    }   }

FIG. 9 is a block diagram of a video processing apparatus 900. Theapparatus 900 may be used to implement one or more of the methodsdescribed herein. The apparatus 900 may be embodied in a smartphone,tablet, computer, Internet of Things (IoT) receiver, and so on. Theapparatus 900 may include one or more processors 902, one or morememories 904 and video processing hardware 906. The processor(s) 902 maybe configured to implement one or more methods described in the presentdocument. The memory (memories) 904 may be used for storing data andcode used for implementing the methods and techniques described herein.The video processing hardware 906 may be used to implement, in hardwarecircuitry, some techniques described in the present document (e.g.,listed in the previous section). In some embodiments, the hardware 906may be partly or entirely within the processors 902, such as a graphicsprocessor.

FIG. 10 shows block diagram of a video encoder.

FIG. 11 is a flowchart for a method 1100 of processing a video. Themethod 1100 includes deriving (1102), for a conversion between a chromablock of a video and a coded representation of the video, parameters ofa cross-component linear model by using downsampled collocatedneighboring top luma samples that are generated from N above neighboringlines of a collocated luma block using a downsampling filter, where N isa positive integer, and performing (1104) the conversion using apredicted chroma block generated using the cross-component linear model.

FIG. 12 is a block diagram of an example video processing system inwhich disclosed techniques may be implemented.

FIG. 12 is a block diagram showing an example video processing system1200 in which various techniques disclosed herein may be implemented.Various implementations may include some or all of the components of thesystem 1200. The system 1200 may include input 1202 for receiving videocontent. The video content may be received in a raw or uncompressedformat, e.g., 8 or 10 bit multi-component pixel values, or may be in acompressed or encoded format. The input 1202 may represent a networkinterface, a peripheral bus interface, or a storage interface. Examplesof network interface include wired interfaces such as Ethernet, passiveoptical network (PON), etc. and wireless interfaces such as Wi-Fi orcellular interfaces.

The system 1200 may include a coding component 1204 that may implementthe various coding or encoding methods described in the presentdocument. The coding component 1204 may reduce the average bitrate ofvideo from the input 1202 to the output of the coding component 1204 toproduce a coded representation of the video. The coding techniques aretherefore sometimes called video compression or video transcodingtechniques. The output of the coding component 1204 may be eitherstored, or transmitted via a communication connected, as represented bythe component 1206. The stored or communicated bitstream (or coded)representation of the video received at the input 1202 may be used bythe component 1208 for generating pixel values or displayable video thatis sent to a display interface 1210. The process of generatinguser-viewable video from the bitstream representation is sometimescalled video decompression. Furthermore, while certain video processingoperations are referred to as “coding” operations or tools, it will beappreciated that the coding tools or operations are used at an encoderand corresponding decoding tools or operations that reverse the resultsof the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface mayinclude universal serial bus (USB) or high definition multimediainterface (HDMI) or Displayport, and so on. Examples of storageinterfaces include SATA (serial advanced technology attachment), PCI,IDE interface, and the like. The techniques described in the presentdocument may be embodied in various electronic devices such as mobilephones, laptops, smartphones or other devices that are capable ofperforming digital data processing and/or video display.

Some embodiments of the disclosed technology include making a decisionor determination to enable a video processing tool or mode. In anexample, when the video processing tool or mode is enabled, the encoderwill use or implement the tool or mode in the processing of a block ofvideo, but may not necessarily modify the resulting bitstream based onthe usage of the tool or mode. That is, a conversion from the block ofvideo to the bitstream representation of the video will use the videoprocessing tool or mode when it is enabled based on the decision ordetermination. In another example, when the video processing tool ormode is enabled, the decoder will process the bitstream with theknowledge that the bitstream has been modified based on the videoprocessing tool or mode. That is, a conversion from the bitstreamrepresentation of the video to the block of video will be performedusing the video processing tool or mode that was enabled based on thedecision or determination.

Some embodiments of the disclosed technology include making a decisionor determination to disable a video processing tool or mode. In anexample, when the video processing tool or mode is disabled, the encoderwill not use the tool or mode in the conversion of the block of video tothe bitstream representation of the video. In another example, when thevideo processing tool or mode is disabled, the decoder will process thebitstream with the knowledge that the bitstream has not been modifiedusing the video processing tool or mode that was enabled based on thedecision or determination.

The disclosed and other solutions, examples, embodiments, modules andthe functional operations described in this document can be implementedin digital electronic circuitry, or in computer software, firmware, orhardware, including the structures disclosed in this document and theirstructural equivalents, or in combinations of one or more of them. Thedisclosed and other embodiments can be implemented as one or morecomputer program products, i.e., one or more modules of computer programinstructions encoded on a computer readable medium for execution by, orto control the operation of, data processing apparatus. The computerreadable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more them. The term “data processing apparatus” encompassesall apparatus, devices, and machines for processing data, including byway of example a programmable processor, a computer, or multipleprocessors or computers. The apparatus can include, in addition tohardware, code that creates an execution environment for the computerprogram in question, e.g., code that constitutes processor firmware, aprotocol stack, a database management system, an operating system, or acombination of one or more of them. A propagated signal is anartificially generated signal, e.g., a machine-generated electrical,optical, or electromagnetic signal, that is generated to encodeinformation for transmission to suitable receiver apparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this document can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random-access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of non-volatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto optical disks; and CD ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in, special purposelogic circuitry.

FIG. 13 is a block diagram that illustrates an example video codingsystem 100 that may utilize the techniques of this disclosure.

As shown in FIG. 13, video coding system 100 may include a source device110 and a destination device 120. Source device 110 generates encodedvideo data which may be referred to as a video encoding device.Destination device 120 may decode the encoded video data generated bysource device 110 which may be referred to as a video decoding device.

Source device 110 may include a video source 112, a video encoder 114,and an input/output (I/O) interface 116.

Video source 112 may include a source such as a video capture device, aninterface to receive video data from a video content provider, and/or acomputer graphics system for generating video data, or a combination ofsuch sources. The video data may comprise one or more pictures. Videoencoder 114 encodes the video data from video source 112 to generate abitstream. The bitstream may include a sequence of bits that form acoded representation of the video data. The bitstream may include codedpictures and associated data. The coded picture is a codedrepresentation of a picture. The associated data may include sequenceparameter sets, picture parameter sets, and other syntax structures. I/Ointerface 116 may include a modulator/demodulator (modem) and/or atransmitter. The encoded video data may be transmitted directly todestination device 120 via I/O interface 116 through network 130 a. Theencoded video data may also be stored onto a storage medium/server 130 bfor access by destination device 120.

Destination device 120 may include an I/O interface 126, a video decoder124, and a display device 122.

I/O interface 126 may include a receiver and/or a modem. I/O interface126 may acquire encoded video data from the source device 110 or thestorage medium/server 130 b. Video decoder 124 may decode the encodedvideo data. Display device 122 may display the decoded video data to auser. Display device 122 may be integrated with the destination device120, or may be external to destination device 120 which be configured tointerface with an external display device.

Video encoder 114 and video decoder 124 may operate according to a videocompression standard, such as the High Efficiency Video Coding (HEVC)standard, Versatile Video Coding (VVC) standard and other current and/orfurther standards.

FIG. 14 is a block diagram illustrating an example of video encoder 200,which may be video encoder 114 in the system 100 illustrated in FIG. 13.

Video encoder 200 may be configured to perform any or all of thetechniques of this disclosure. In the example of FIG. 14, video encoder200 includes a plurality of functional components. The techniquesdescribed in this disclosure may be shared among the various componentsof video encoder 200. In some examples, a processor may be configured toperform any or all of the techniques described in this disclosure.

The functional components of video encoder 200 may include a partitionunit 201, a predication unit 202 which may include a mode select unit203, a motion estimation unit 204, a motion compensation unit 205 and anintra prediction unit 206, a residual generation unit 207, a transformunit 208, a quantization unit 209, an inverse quantization unit 210, aninverse transform unit 211, a reconstruction unit 212, a buffer 213, andan entropy encoding unit 214.

In other examples, video encoder 200 may include more, fewer, ordifferent functional components. In an example, predication unit 202 mayinclude an intra block copy (IBC) unit. The IBC unit may performpredication in an IBC mode in which at least one reference picture is apicture where the current video block is located.

Furthermore, some components, such as motion estimation unit 204 andmotion compensation unit 205 may be highly integrated, but arerepresented in the example of FIG. 14 separately for purposes ofexplanation.

Partition unit 201 may partition a picture into one or more videoblocks. Video encoder 200 and video decoder 300 may support variousvideo block sizes.

Mode select unit 203 may select one of the coding modes, intra or inter,e.g., based on error results, and provide the resulting intra- orinter-coded block to a residual generation unit 207 to generate residualblock data and to a reconstruction unit 212 to reconstruct the encodedblock for use as a reference picture. In some example, Mode select unit203 may select a combination of intra and inter predication (CIIP) modein which the predication is based on an inter predication signal and anintra predication signal. Mode select unit 203 may also select aresolution for a motion vector (e.g., a sub-pixel or integer pixelprecision) for the block in the case of inter-predication.

To perform inter prediction on a current video block, motion estimationunit 204 may generate motion information for the current video block bycomparing one or more reference frames from buffer 213 to the currentvideo block. Motion compensation unit 205 may determine a predictedvideo block for the current video block based on the motion informationand decoded samples of pictures from buffer 213 other than the pictureassociated with the current video block.

Motion estimation unit 204 and motion compensation unit 205 may performdifferent operations for a current video block, for example, dependingon whether the current video block is in an I slice, a P slice, or a Bslice.

In some examples, motion estimation unit 204 may perform uni-directionalprediction for the current video block, and motion estimation unit 204may search reference pictures of list 0 or list 1 for a reference videoblock for the current video block. Motion estimation unit 204 may thengenerate a reference index that indicates the reference picture in list0 or list 1 that contains the reference video block and a motion vectorthat indicates a spatial displacement between the current video blockand the reference video block. Motion estimation unit 204 may output thereference index, a prediction direction indicator, and the motion vectoras the motion information of the current video block. Motioncompensation unit 205 may generate the predicted video block of thecurrent block based on the reference video block indicated by the motioninformation of the current video block.

In other examples, motion estimation unit 204 may perform bi-directionalprediction for the current video block, motion estimation unit 204 maysearch the reference pictures in list 0 for a reference video block forthe current video block and may also search the reference pictures inlist 1 for another reference video block for the current video block.Motion estimation unit 204 may then generate reference indexes thatindicate the reference pictures in list 0 and list 1 containing thereference video blocks and motion vectors that indicate spatialdisplacements between the reference video blocks and the current videoblock. Motion estimation unit 204 may output the reference indexes andthe motion vectors of the current video block as the motion informationof the current video block. Motion compensation unit 205 may generatethe predicted video block of the current video block based on thereference video blocks indicated by the motion information of thecurrent video block.

In some examples, motion estimation unit 204 may output a full set ofmotion information for decoding processing of a decoder.

In some examples, motion estimation unit 204 may do not output a fullset of motion information for the current video. Rather, motionestimation unit 204 may signal the motion information of the currentvideo block with reference to the motion information of another videoblock. For example, motion estimation unit 204 may determine that themotion information of the current video block is sufficiently similar tothe motion information of a neighboring video block.

In one example, motion estimation unit 204 may indicate, in a syntaxstructure associated with the current video block, a value thatindicates to the video decoder 300 that the current video block has thesame motion information as the another video block.

In another example, motion estimation unit 204 may identify, in a syntaxstructure associated with the current video block, another video blockand a motion vector difference (MVD). The motion vector differenceindicates a difference between the motion vector of the current videoblock and the motion vector of the indicated video block. The videodecoder 300 may use the motion vector of the indicated video block andthe motion vector difference to determine the motion vector of thecurrent video block.

As discussed above, video encoder 200 may predictively signal the motionvector. Two examples of predictive signaling techniques that may beimplemented by video encoder 200 include advanced motion vectorpredication (AMVP) and merge mode signaling.

Intra prediction unit 206 may perform intra prediction on the currentvideo block. When intra prediction unit 206 performs intra prediction onthe current video block, intra prediction unit 206 may generateprediction data for the current video block based on decoded samples ofother video blocks in the same picture. The prediction data for thecurrent video block may include a predicted video block and varioussyntax elements.

Residual generation unit 207 may generate residual data for the currentvideo block by subtracting (e.g., indicated by the minus sign) thepredicted video block(s) of the current video block from the currentvideo block. The residual data of the current video block may includeresidual video blocks that correspond to different sample components ofthe samples in the current video block.

In other examples, there may be no residual data for the current videoblock for the current video block, for example in a skip mode, andresidual generation unit 207 may not perform the subtracting operation.

Transform processing unit 208 may generate one or more transformcoefficient video blocks for the current video block by applying one ormore transforms to a residual video block associated with the currentvideo block.

After transform processing unit 208 generates a transform coefficientvideo block associated with the current video block, quantization unit209 may quantize the transform coefficient video block associated withthe current video block based on one or more quantization parameter (QP)values associated with the current video block.

Inverse quantization unit 210 and inverse transform unit 211 may applyinverse quantization and inverse transforms to the transform coefficientvideo block, respectively, to reconstruct a residual video block fromthe transform coefficient video block. Reconstruction unit 212 may addthe reconstructed residual video block to corresponding samples from oneor more predicted video blocks generated by the predication unit 202 toproduce a reconstructed video block associated with the current blockfor storage in the buffer 213.

After reconstruction unit 212 reconstructs the video block, loopfiltering operation may be performed reduce video blocking artifacts inthe video block.

Entropy encoding unit 214 may receive data from other functionalcomponents of the video encoder 200. When entropy encoding unit 214receives the data, entropy encoding unit 214 may perform one or moreentropy encoding operations to generate entropy encoded data and outputa bitstream that includes the entropy encoded data.

FIG. 15 is a block diagram illustrating an example of video decoder 300which may be video decoder 114 in the system 100 illustrated in FIG. 13.

The video decoder 300 may be configured to perform any or all of thetechniques of this disclosure. In the example of FIG. 15, the videodecoder 300 includes a plurality of functional components. Thetechniques described in this disclosure may be shared among the variouscomponents of the video decoder 300. In some examples, a processor maybe configured to perform any or all of the techniques described in thisdisclosure.

In the example of FIG. 15, video decoder 300 includes an entropydecoding unit 301, a motion compensation unit 302, an intra predictionunit 303, an inverse quantization unit 304, an inverse transformationunit 305, and a reconstruction unit 306 and a buffer 307. Video decoder300 may, in some examples, perform a decoding pass generally reciprocalto the encoding pass described with respect to video encoder 200 (e.g.,FIG. 14).

Entropy decoding unit 301 may retrieve an encoded bitstream. The encodedbitstream may include entropy coded video data (e.g., encoded blocks ofvideo data). Entropy decoding unit 301 may decode the entropy codedvideo data, and from the entropy decoded video data, motion compensationunit 302 may determine motion information including motion vectors,motion vector precision, reference picture list indexes, and othermotion information. Motion compensation unit 302 may, for example,determine such information by performing the AMVP and merge mode.

Motion compensation unit 302 may produce motion compensated blocks,possibly performing interpolation based on interpolation filters.Identifiers for interpolation filters to be used with sub-pixelprecision may be included in the syntax elements.

Motion compensation unit 302 may use interpolation filters as used byvideo encoder 20 during encoding of the video block to calculateinterpolated values for sub-integer pixels of a reference block. Motioncompensation unit 302 may determine the interpolation filters used byvideo encoder 200 according to received syntax information and use theinterpolation filters to produce predictive blocks.

Motion compensation unit 302 may uses some of the syntax information todetermine sizes of blocks used to encode frame(s) and/or slice(s) of theencoded video sequence, partition information that describes how eachmacroblock of a picture of the encoded video sequence is partitioned,modes indicating how each partition is encoded, one or more referenceframes (and reference frame lists) for each inter-encoded block, andother information to decode the encoded video sequence.

Intra prediction unit 303 may use intra prediction modes for examplereceived in the bitstream to form a prediction block from spatiallyadjacent blocks. Inverse quantization unit 303 inverse quantizes, i.e.,de-quantizes, the quantized video block coefficients provided in thebitstream and decoded by entropy decoding unit 301. Inverse transformunit 303 applies an inverse transform.

Reconstruction unit 306 may sum the residual blocks with thecorresponding prediction blocks generated by motion compensation unit202 or intra-prediction unit 303 to form decoded blocks. If desired, adeblocking filter may also be applied to filter the decoded blocks inorder to remove blockiness artifacts. The decoded video blocks are thenstored in buffer 307, which provides reference blocks for subsequentmotion compensation.

Some embodiments of the disclosed technology include making a decisionor determination to enable a video processing tool or mode. In anexample, when the video processing tool or mode is enabled, the encoderwill use or implement the tool or mode in the processing of a block ofvideo, but may not necessarily modify the resulting bitstream based onthe usage of the tool or mode. That is, a conversion from the block ofvideo to the bitstream representation of the video will use the videoprocessing tool or mode when it is enabled based on the decision ordetermination. In another example, when the video processing tool ormode is enabled, the decoder will process the bitstream with theknowledge that the bitstream has been modified based on the videoprocessing tool or mode. That is, a conversion from the bitstreamrepresentation of the video to the block of video will be performedusing the video processing tool or mode that was enabled based on thedecision or determination.

In the present document, the term “video processing” may refer to videoencoding, video decoding, video compression or video decompression. Forexample, video compression algorithms may be applied during conversionfrom pixel representation of a video to a corresponding bitstreamrepresentation or vice versa. The bitstream representation of a currentvideo block may, for example, correspond to bits that are eitherco-located or spread in different places within the bitstream, as isdefined by the syntax. For example, a macroblock may be encoded in termsof transformed and coded error residual values and also using bits inheaders and other fields in the bitstream.

A listing of clauses preferred by some embodiments is provided next.

The first set of clauses show example embodiments of techniquesdiscussed in the previous sections.

1. A method of video processing, comprising: deriving, for a conversionbetween a chroma block of a video and a coded representation of thevideo, parameters of a cross-component linear model by using downsampledcollocated neighboring top luma samples that are generated from N aboveneighboring lines of a collocated luma block using a downsamplingfilter, where N is a positive integer; and performing the conversionusing a predicted chroma block generated using the cross-componentlinear model.

2. The method of clause 1, wherein, due to the chroma block not being ata top coding tree unit boundary, the N above neighboring linescorrespond to a nearest above line of the collocated luma block.

3. The method of any of clauses 1-2, wherein the downsampling filter isalso applied for generating downsampled collocated neighbouring leftluma samples.

4. The method of any of clauses 1-2, wherein the downsampling filter isdifferent from another downsampling filter used for generatingdownsampled collocated neighbouring left luma samples.

5. The method of any of clauses 1, wherein the downsampling filter isindependent of a position of the chroma block with respect to a topboundary of the coding tree unit.

6. The method of any of clauses 1, wherein the method is selectivelyapplied due to the video having a 4:2:2 format.

7. The method of clause 1, wherein N is greater than one.

8. The method of clause 7, wherein the N above neighboring lines includethe nearest above line and a second nearest above line.

9. The method of clause 1, wherein the downsampling filter is dependenton a color format of the video.

10. The method of any of clauses 1-9, wherein the downsampling filter isa 6-tap filter.

11. The method of any of clauses 1-9, wherein the downsampling filter isa 5-tap filter.

12. The method of any of clauses 1 to 11, wherein the conversioncomprises encoding the video into the coded representation.

13. The method of any of clauses 1 to 11, wherein the conversioncomprises decoding the coded representation to generate pixel values ofthe video.

14. A video decoding apparatus comprising a processor configured toimplement a method recited in one or more of clauses 1 to 13.

15. A video encoding apparatus comprising a processor configured toimplement a method recited in one or more of clauses 1 to 13.

16. A computer program product having computer code stored thereon, thecode, when executed by a processor, causes the processor to implement amethod recited in any of clauses 1 to 13.

17. A method, apparatus or system described in the present document.

The second set of clauses describe certain features and aspects of thedisclosed techniques in the previous section (e.g., item 1).

1. A method of video processing, comprising: deriving 1602, foraconversion between a chroma block of a video and a bitstreamrepresentation of the video, parameters of a cross-component linearmodel by using downsampled luma samples that are generated from N aboveneighboring lines of a collocated luma block of the chroma block using adownsampling filter, where N is a positive integer; and performing theconversion using a predicted chroma block generated using thecross-component linear model.

2. The method of clause 1, wherein, due to the chroma block not being ata top coding tree unit boundary, the N above neighboring linescorrespond to a nearest above line of the collocated luma block.

3. The method of any of clauses 1-2, wherein the downsampling filter isalso applied for generating other downsampled luma samples that aregenerated from left neighboring lines of the collocated luma block.

4. The method any of clauses 1-2, wherein another downsampling filter isapplied for generating other downsampled luma samples that are generatedfrom left neighboring lines of the collocated luma block.

5. The method of any of clauses 1-4, wherein the downsampling filter hasfilter coefficients of [1, 2, 1].

6. The method of any of clauses 1-5, wherein a downsampled luma sample,pDsY[x], satisfies an equation,pDsY[x]=(pY[2*x−1][4]+2*pY[2*x][−1]+pY[2*x+1][4]+2)>>2, whereinpY[2*x][−1], pY[2*x−1][4] and pY[2*x+1][4] are luma samples from thenearest above neighboring line, x being an integer.

7. The method of any of clauses 1-6, wherein the downsampling filter isindependent of a position of the chroma block with respect to a topboundary of the coding tree unit.

8. The method of any of clauses 1-6, wherein the method is selectivelyapplied due to a 4:2:2 colour format of the video.

9. The method of clause 1, wherein, due to the chroma block not being ata top coding tree unit boundary, the N above neighboring lines include anearest above line of the collocated luma block but excludes a secondnearest above line.

10. The method of clause 1, wherein N is greater than one.

11. The method of clause 10, wherein the N above neighboring linesinclude the nearest above line and a second nearest above line.

12. The method of clause 1, wherein the downsampling filter is dependenton a color format of the video.

13. The method of any of clause 1-12, wherein the downsampling filter isa 6-tap filter.

14. The method of any of clauses 1-12, wherein the downsampling filteris a 5-tap filter.

15. The method of any of clauses 1 to 14, wherein the conversionincludes encoding the video into the bitstream representation.

16. The method of any of clauses 1 to 14, wherein the conversionincludes decoding the video from the bitstream representation.

17. A video processing apparatus comprising a processor configured toimplement a method recited in any one or more of clauses 1 to 16.

18. A computer readable medium storing program code that, when executed,causes a processor to implement a method recited in any one or more ofclauses 1 to 16.

19. A computer readable medium that stores a bitstream representationgenerated according to any of the above described methods.

The third set of clauses describe certain features and aspects of thedisclosed techniques in the previous sections (e.g., items 2 to 7).

1. A method of video processing (e.g., method 1610 as shown in FIG.16A), comprising: determining 1612, for a conversion between a videoregion of a component of a video and a bitstream representation of thevideo, a maximum allowed block size for a video block coded using atransform skip mode; and performing 1614 the conversion based on thedetermining.

2. The method of clause 1, wherein the transform skip mode comprises,during encoding, coding residual of the video block without applying anon-identity transform, or during decoding, determining a decoded videoblock without applying a non-identity inverse transform to residualscoded in the bitstream representation.

3. The method of clause 1, wherein the transform skip mode comprises aBDPCM (block differential pulse-code modulation) that corresponds to anintra-coding tool that uses a differential pulse-code modulation (DPCM)at a block level.

4. The method of clause 1, wherein the maximum allowed block size isdependent on whether the transform skipped block is chroma block or aluma block.

5. The method of clause 1, wherein the maximum allowed block size isdependent on a chroma component of the transform skipped block.

6. The method of clause 1, wherein a maximum allowed block size for aluma block (MaxTsSizeY) and a maximum allowed block size fora chromablock (MaxTsSizeC) are separately signaled in the bitstreamrepresentation.

7. The method of clause 6, wherein the MaxTsSizeC and/or the MaxTsSizeYis signaled at a sequence level, a picture level, a slice level, or atile group level.

8. The method of clause 6, wherein the MaxTsSizeY is conditionallysignaled based on an enablement status of the transform skip mode.

9. The method of clause 6, wherein the MaxTsSizeY is conditionallysignaled based on a color format and/or an enablement status of thetransform skip mode.

10. The method of clause 1, wherein the conversion is performed byutilizing a predictive coding between maximum block sizes for a lumacomponent and a chroma component.

11. The method of clause 1, wherein the video block is a chroma videoblock and wherein the maximum allowed block size (MaxTsSizeC) for thevideo block is dependent on a maximum allowed block size (MaxTsSizeY)for another video block of a luma component.

12. The method of clause 11, wherein the MaxTsSizeC is set equal to theMaxTsSizeY.

13. The method of clause 11, wherein the MaxTsSizeC is set equal toMaxTsSizeY/N, whereby N is an integer.

14. The method of clause 1, wherein the video block is a chroma videoblock and wherein the maximum allowed block size (MaxTsSizeC) for thevideo block is set according to chroma sub sampling ratios.

15. The method of clause 14, wherein the MaxTsSizeC is set equal to i)MaxTsSizeY>>SubWidthC, ii) MaxTsSizeY>>SubHeightC, iii) MaxTsSizeY>>max(SubWidthC, SubHeightC), iv) MaxTsSizeY>>min (SubWidthC, SubHeightC),wherein MaxTsSiZeY indicates a maximum block size of a luma video blockand SubWidthC and SubHeightC are predefined.

16. A method of video processing (e.g., method 1610 as shown in FIG.16A), comprising: performing a conversion between a video comprisingvideo blocks and a bitstream representation of the video according to afirst rule and a second rule. A transform skip coding tool is used forcoding a first portion of the video blocks, a transform coding tool isused for coding a second portion of the video blocks, the first rulespecifies a maximum allowed block size for the first portion of thevideo blocks and the second rule specifies a maximum allowed block sizefor the second portion of the video blocks, and the maximum allowedblock size for the first portion of the video blocks is different formthe maximum allowed block size for the second portion of the videoblock.

17. The method of clause 16, wherein the maximum allowed block sizecorresponds to a width and a height of a corresponding block.

18. The method of clause 17, wherein the width and the height of themaximum allowed block size are signaled separately.

19. The method of clause 17, wherein, for the second portion of thevideo blocks that are chroma blocks, the width (MaxTsSizeWC) is setequal to MaxTsSizeY>>SubWidthC and the height (MaxTsSizeHC) is set equalto MaxTsSizeY>>SubHeightC, wherein MaxTsSizeY indicates a maximumallowed block size for a luma block.

20. A method of video processing (e.g., method 1610 as shown in FIG.16A), comprising: performing a conversion between a video comprising oneor more chroma blocks and a bitstream representation of the video. Thebitstream representation conforms to a format rule that specifies thatwhether a syntax element to indicate usage of a transform skip tool isincluded in the bitstream representation depends on a maximum allowedsize for a chroma block that is coded using the transform skip tool.

21. The method of clause 20, wherein the transform skip tool includesbypassing a transform or applying an identity transform.

22. The method of clause 20, wherein the syntax element is signaled in acase that tbW is less than or equal to MaxTsSizeC and tbH is less thanor equal to MaxTsSizeC, wherein tbW and tbH are a width and a height ofthe chroma block, respectively, and MaxTsSizeC is the maximum allowedsize for the chroma block, respectively.

23. The method of clause 20, wherein the syntax element is signaled in acase that tbW is less than or equal to MaxTsSizeWC and tbH is less thanor equal to MaxTsSizeHC, wherein tbW and tbH are a width and a height ofthe chroma block, respectively, and MaxTsSizeWC and MaxTsSizeHCrepresent a width and a height of the maximum allowed size for thechroma block, respectively.

24. The method of clause 20, wherein the transform skip tool includes aBDPCM (block differential pulse-code modulation) mode that correspondsto an intra-coding tool that uses a differential pulse-code modulation(DPCM) at a block level.

25. A method of video processing (e.g., method 1610 as shown in FIG.16A), comprising: performing a conversion between a video comprising oneor more first video blocks of a first chroma component and one or moresecond video blocks of a second chroma component and a bitstreamrepresentation of the video. The bitstream representation conforms to aformat rule that specifies to use a syntax element that jointlyindicates availability of a transform skip tool for coding the one ormore first chroma blocks and the one or more second chroma blocks.

26. The method of clause 25, wherein the syntax element has a binaryvalue.

27. The method of clause 25, wherein the transform skip tool is enabledor disabled in the one or more first video blocks and the one or moresecond video blocks according to the syntax element.

28. The method of clause 25, wherein the format rule further specifiesthat an additional syntax element is included in the bitstreamrepresentation based on whether a value of the syntax element is equalto K, whereby K is an integer.

29. The method of clause 28, wherein the second syntax element is usedto indicate to which one of the one or more first video blocks and theone or more second video blocks the transform skip tool is applied.

30. The method of clause 25, wherein the syntax element has a non-binaryvalue.

31. The method of clause 30, wherein the syntax element is coed with afixed length, unary, truncated unary, or k-th order Exp-Golomb (EG)binarization method.

32. The method of clause 25, wherein the syntax element is context codedor bypass coded.

33. The method of any one of previous clauses, wherein whether to and/orhow to apply the method is signaled at a sequence level, a picturelevel, a slice level, or a tile group level.

34. The method of any one of previous clauses, wherein the method isfurther based on coded information.

35. The method of any of clauses 1 to 34, wherein the conversionincludes encoding the video into the bitstream representation.

36. The method of any of clauses 1 to 34, wherein the conversionincludes decoding the video from the bitstream representation.

37. A video processing apparatus comprising a processor configured toimplement a method recited in any one or more of clauses 1 to 36.

38. A computer readable medium storing program code that, when executed,causes a processor to implement a method recited in any one or more ofclauses 1 to 36.

39. A computer readable medium that stores a coded representation or abitstream representation generated according to any of the abovedescribed methods.

While this patent document contains many specifics, these should not beconstrued as limitations on the scope of any subject matter or of whatmay be claimed, but rather as descriptions of features that may bespecific to particular embodiments of particular techniques. Certainfeatures that are described in this patent document in the context ofseparate embodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a sub combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

The invention claimed is:
 1. A method of video processing, comprising:determining, for a first conversion between a chroma block of a videoand a bitstream of the video, that a prediction mode is applied to thechroma block, wherein in the prediction mode, prediction samples of thechroma block are derived based on reconstructed luma samples of acollocated luma block of the chroma block; deriving parameters of theprediction mode based on neighbouring chroma samples of the chroma blockand downsampled neighbouring top luma samples of the collocated lumablock; and performing the first conversion based on the parameters,wherein in case that the chroma block has a color format of 4:2:2, thedownsampled neighbouring top luma samples are derived based on a nearestabove neighbouring line of the collocated luma block; wherein in casethat a chroma collocated flag is equal to 0, the chroma block is not ata top coding tree unit boundary and the chroma block has a color formatof 4:2:0, (pY[SubWidthC*x−1][−1]+pY[SubWidthC*x−1][−2]+2*pY[SubWidthC*x][−1]+2*pY[SubWidthC*x][−2]+pY[SubWidthC*x+1][−1]+pY[SubWidthC*x+1][−2]+4)>>3are used to derive at lease one of downsampled neighbouring top lumasamples, wherein pY[SubWidthC*x−1 ][−1], pY[SubWidthC*x−1][−2],pY[SubWidthC*x][−1], pY[SubWidthC*x][−2], pY[SubWidthC*x+1][−1] andpY[SubWidthC*x+1][−2] denote luma samples from neighboring lines of thecollocated luma block, and wherein x is an integer and SubWidthC isequal to 2 in case that the chroma block has a color format of 4:2:0. 2.The method of claim 1, wherein in response to the chroma block having acolor format of 4:2:2, a second above neighbouring line of thecollocated luma block is excluded from being used to derive thedownsampled neighbouring top luma samples.
 3. The method of claim 1,wherein in response to the chroma block having a color format of 4:2:2,a same downsampling filter is used to derive the downsampledneighbouring top luma samples regardless of whether the chroma block isat the top coding tree unit boundary or not.
 4. The method of claim 3,wherein in response to the chroma block being at the top coding treeunit boundary, the downsampled neighbouring top luma samples are derivedbased on the nearest above neighbouring line of the collocated lumablock.
 5. The method of claim 4, wherein in response to the chroma blockbeing at the top coding tree unit boundary or the chroma block having acolor format of 4:2:2,pDsY[x]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2, whereinpDsY[x] denotes a downsampled neighbouring top luma sample, and whereinpY[2*x][−1], pY[2*x−1][−1] and pY[2*x+1][−1] denotes luma samples fromthe nearest above neighboring line of the collocated luma block.
 6. Themethod of claim 1, wherein the parameters of the prediction mode arefurther derived based on downsampled neighbouring left luma samples ofthe collocated luma block; wherein the downsampled neighbouring top lumasamples and the downsampled neighbouring left luma samples are derivedusing downsampling filters with same filter coefficients in response tothe chroma block having a color format of 4:2:2.
 7. The method of claim6, wherein the same filter coefficients are [1 2 1].
 8. The method ofclaim 1, wherein a first syntax element specifying a maximum block sizeused for a transform skip mode is conditionally included in thebitstream based on a value of a transform skip enabled flag included ina sequence parameter set in the bitstream.
 9. The method of claim 1,wherein the conversion includes decoding the video from the bitstream.10. The method of claim 1, wherein the conversion includes encoding thevideo into the bitstream.
 11. An apparatus for processing video datacomprising a processor and a non-transitory memory with instructionsthereon, wherein the instructions upon execution by the processor, causethe processor to: determine, for a first conversion between a chromablock of a video and a bitstream of the video, that a prediction mode isapplied to the chroma block, wherein in the prediction mode, predictionsamples of the chroma block are derived based on reconstructed lumasamples of a collocated luma block of the chroma block; deriveparameters of the prediction mode based on neighbouring chroma samplesof the chroma block and downsampled neighbouring top luma samples of thecollocated luma block; and perform the first conversion based on theparameters, wherein in case that the chroma block having has a colorformat of 4:2:2, the downsampled neighbouring top luma samples arederived based on a nearest above neighbouring line of the collocatedluma block; wherein in case that a chroma collocated flag is equal to 0,the chroma block is not at a top coding tree unit boundary and thechroma block has a color format of 4:2:0,(pY[SubWidthC*x−1][−1]+pY[SubWidthC*x−1][−2]+2*pY[SubWidthC*x][−1]+2*pY[SubWidthC*x][−2]+pY[SubWidthC*x+1][−1]+pY[SubWidthC*x+1][−2]+4)>>3are used to derive at lease one of downsampled neighbouring top lumasamples, wherein pY[SubWidthC*x−1 ][−1], pY[SubWidthC*x−1][−2],pY[SubWidthC*x][−1], pY[SubWidthC*x][−2], pY[SubWidthC*x+1][−1] andpY[SubWidthC*x+1][−2] denote luma samples from neighboring lines of thecollocated luma block, and wherein x is an integer and SubWidthC isequal to 2 in case that the chroma block has a color format of 4:2:0.12. The apparatus of claim 11, wherein in response to the chroma blockhaving a color format of 4:2:2, a second above neighbouring line of thecollocated luma block is excluded from being used to derive thedownsampled neighbouring top luma samples.
 13. The apparatus of claim11, wherein in response to the chroma block having a color format of4:2:2, a same downsampling filter is used to derive the downsampledneighbouring top luma samples regardless of whether the chroma block isat the top coding tree unit boundary or not.
 14. The apparatus of claim13, wherein in response to the chroma block being at the top coding treeunit boundary, the downsampled neighbouring top luma samples are derivedbased on the nearest above neighbouring line of the collocated lumablock.
 15. The apparatus of claim 14, wherein in response to the chromablock being at the top coding tree unit boundary or the chroma blockhaving a color format of 4:2:2,pDsY[x]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2, whereinpDsY[x] denotes a downsampled neighbouring top luma sample, and whereinpY[2*x][−1], pY[2*x−1][−1] and pY[2*x+1][−1] denotes luma samples fromthe nearest above neighboring line of the collocated luma block.
 16. Theapparatus of claim 11, wherein the parameters of the prediction mode arefurther derived based on downsampled neighbouring left luma samples ofthe collocated luma block; wherein the downsampled neighbouring top lumasamples and the downsampled neighbouring left luma samples are derivedusing downsampling filters with same filter coefficients in response tothe chroma block having a color format of 4:2:2.
 17. The apparatus ofclaim 16, wherein the same filter coefficients are [1 2 1].
 18. Anon-transitory computer-readable storage medium storing instructionsthat cause a processor to: determine, for a first conversion between achroma block of a video and a bitstream of the video, that a predictionmode is applied to the chroma block, wherein in the prediction mode,prediction samples of the chroma block are derived based onreconstructed luma samples of a collocated luma block of the chromablock; derive parameters of the prediction mode based on neighbouringchroma samples of the chroma block and downsampled neighbouring top lumasamples of the collocated luma block; and perform the first conversionbased on the parameters, wherein in case that the chroma block has acolor format of 4:2:2, the downsampled neighbouring top luma samples arederived based on a nearest above neighbouring line of the collocatedluma block; wherein in case that a chroma collocated flag is equal to 0,the chroma block is not at a top coding tree unit boundary and thechroma block has a color format of 4:2:0,(pY[SubWidthC*x−1][−1]+pY[SubWidthC*x−1][−2]+2*pY[SubWidthC*x][−1]+2*pY[SubWidthC*x][−2]+pY[SubWidthC*x+1][−1]+pY[SubWidthC*x+1][−2]+4)>>3are used to derive at lease one of downsampled neighbouring top lumasamples, wherein pY[SubWidthC*x−1 ][−1], pY[ SubWidthC*x−1][−2],pY[SubWidthC*x][−1], pY[SubWidthC*x][−2], pY[SubWidthC*x+1][−1] andpY[SubWidthC*x+1][−2] denote luma samples from neighboring lines of thecollocated luma block, and wherein x is an integer and SubWidthC isequal to 2 in case that the chroma block has a color format of 4:2:0.19. A method of storing a bitstream of a video which is generated by avideo processing apparatus to a non-transitory computer readablerecording medium, the method comprising: determining, for a firstconversion between a chroma block of the video and a bitstream of thevideo, that a prediction mode is applied to the chroma block, wherein inthe prediction mode, prediction samples of the chroma block are derivedbased on reconstructed luma samples of a collocated luma block of thechroma block; deriving parameters of the prediction mode based onneighbouring chroma samples of the chroma block and downsampledneighbouring top luma samples of the collocated luma block; and storingthe bits tre am to the non-transitory computer readable recordingmedium, wherein in case that the chroma block has a color format of4:2:2, the downsampled neighbouring top luma samples are derived basedon a nearest above neighbouring line of the collocated luma block;wherein in case that a chroma collocated flag is equal to 0, the chromablock is not at a top coding tree unit boundary and the chroma block hasa color format of 4:2:0, (pY[SubWidthC*x−1][−1]+pY[SubWidthC*x−1][−2]+2*pY[SubWidthC*x][−1]+2*pY[SubWidthC*x][−2]+pY[SubWidthC*x+1][−1]+pY[SubWidthC*x+1][−2]+4)>>3are used to derive at lease one of downsampled neighbouring top lumasamples, wherein pY[SubWidthC*x−1 ][−1], pY[SubWidthC*x−1][−2],pY[SubWidthC*x][−1], pY[SubWidthC*x][−2], pY[SubWidthC*x+1][−1] andpY[SubWidthC*x+1][−2] denote luma samples from neighboring lines of thecollocated luma block, and wherein x is an integer and SubWidthC isequal to 2 in case that the chroma block has a color format of 4:2:0.20. The apparatus of claim 11, wherein a first syntax element specifyinga maximum block size used for a transform skip mode is conditionallyincluded in the bitstream based on a value of a transform skip enabledflag included in a sequence parameter set in the bitstream.